The signed Kolmogorov-Smirnov test: why it should not be used
نویسنده
چکیده
The two-sample Kolmogorov-Smirnov (KS) test is often used to decide whether two random samples have the same statistical distribution. A popular modification of the KS test is to use a signed version of the KS statistic to infer whether the values of one sample are statistically larger than the values of the other. The underlying hypotheses of the KS test are intrinsically incompatible with this approach and the test can produce false positives supported by extremely low p-values. This potentially makes the signed KS test a tool of p-hacking, which should be discouraged by replacing it with standard tests such as the t-test and by providing confidence intervals instead of p-values.
منابع مشابه
Application of the Kolmogorov-Smirnov Test to Estimate the Threshold When Estimating the Extreme Value Index
The Pareto distribution model assumption in the peaks over threshold method, will be tested by making using of the Kolmogorov-Smirnov goodness of fit method. Pareto distributed variables can be transformed to exponential, and the test will be for exponentiality. It was found that the statistic can be used as an indication of where to choose the threshold and to check the Pareto model assumption.
متن کاملA Modified Kolmogorov-Smirnov Test for Normality
In this paper we propose an improvement of the Kolmogorov-Smirnov test for normality. In the current implementation of the Kolmogorov-Smirnov test, a sample is compared with a normal distribution where the sample mean and the sample variance are used as parameters of the distribution. We propose to select the mean and variance of the normal distribution that provide the closest fit to the data....
متن کاملAn Exploration of the Kolmogorov-Smirnov Test as Competitor to Mutual Information Analysis
A theme of recent side-channel research has been the quest for distinguishers which remain e ective even when few assumptions can be made about the underlying distribution of the measured leakage traces. The Kolmogorov-Smirnov (KS) test is a well known non-parametric method for distinguishing between distributions, and, as such, a perfect candidate and an interesting competitor to the (already ...
متن کاملمقایسه کیفیت زندگی افراد سالمند ورزشکار و غیرورزشکار شهر تبریز
Objectives: This study aimed to assess the quality of life in the athletes and non-athletes elderly and were compared between them. Methods & Materials: This research based on the purpose is applied and based on the method is descriptive. 189 elderly people were selected as sample was used of Tabriz city and for sampling the random method. Data collection tool was demographic questionnaire a...
متن کاملPower comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests
The importance of normal distribution is undeniable since it is an underlying assumption of many statistical procedures such as t-tests, linear regression analysis, discriminant analysis and Analysis of Variance (ANOVA). When the normality assumption is violated, interpretation and inferences may not be reliable or valid. The three common procedures in assessing whether a random sample of indep...
متن کامل